Got it, thanks Max. If I understand correctly, you are proposing each Transform in Klay Beam could have its own Dataclass that has the exact properties relevant to that transform. For example LoadWithTorchaudio could output data classes that look like this:

@dataclass(frozen=True)
class TimeSeriesTorchData:
    source_dir: str
    source_path: str
    sample_rate: int
    tensor: torch.Tensor

This is still somewhat resuable–we could write a .npy loader that outputs this type. Also downstream PTransforms don't have to assert element.source_dir etc. Is that what you had in mind?

One of the problems that I want to solve is that Transforms should be able to forward little bits of metadata from their input elements to subsequent transforms in the pipeline. One example is SkipCompleted, which really benefits from knowing the expected shape of the output file(s).

I would like Transforms (like SkipCompleted and LoadWithX) that sit between the main file matching transform, and the final "persistence" transform to be able to forward these little bits of metadata when they are available. This means that we can setup just one function that prescribes the the composition of the expected input and output filetypes and extensions. Then all core transforms can forward any available metadata to their outputs, even when they do not necessarily use that data.

If every Transform in the pipeline has rigid input and output types, I'm worried that:

1. We have to maintain many slightly different dataclasses making it a chore to find the right one when writing transforms
2. It becomes difficult to write Transforms that forward little bits of metadata that they do not necessarily use, but MAY be useful for subsequent transforms. One solution is to add a .metadata dicts to all Transform IO dataclasses, but then we have to shuffle these bits of data between properties and the dictionary, and that feels error-prone.

Let's think together if there is a better way to accomplish these goals while also maintaining more type safety than the current proposal. Maybe the answer is to just start writing the updated transforms and pipelines, and see what approach goes smoother.